Robust Markov Decision Processes

Markov decision processes (MDPs) are powerful tools for decision making in uncertain dynamic environments. However, the solutions of MDPs are of limited practical use due to their sensitivity to distributional model parameters, which are typically unknown and have to be estimated by the decision maker. To counter the detrimental effects of estimation errors, we consider robust MDPs that offer probabilistic guarantees in view of the unknown parameters. To this end, we assume that an observation history of the MDP is available. Based on this history, we derive a confidence region that contains the unknown parameters with a pre-specified probability 1-ß. Afterwards, we determine a policy that attains the highest worst-case performance over this confidence region. By construction, this policy achieves or exceeds its worst-case performance with a confidence of at least 1 - ß. Our method involves the solution of tractable conic programs of moderate size.

Robust optimal decisions with imprecise forecasts

A robust minimax approach for optimal investment decisions with imprecise return forecasts and risk estimations in financial portfolio management is considered. Single-period and multi-period mean-variance optimization models are extended to worst-case design with multiple rival risk estimations and return forecasts. In multi-period stochastic formulation of classical mean-variance portfolio optimization problem, an investor makes an investment decision based on expectations and/or scenarios up to some intermediate times prior to the horizon and, consequently, rebalances or restructures the portfolio. Multi-period portfolio optimization entails the construction of a scenario tree representing a discretized estimate of uncertainties and associated probabilities in future stages. It is well known that return forecasts and risk estimations are inherently inaccurate and there are different rival estimates, or scenario trees. Robust optimization models are presented and imprecise nature of moment forecasts to reduce the risk of making a decision based on the wrong scenario is addressed. The worst-case performance is guaranteed in view of all rival risk and return scenarios and will only improve when any scenario other than the worst-case is realized. The ex-ante performance of minimax models is tested using historical data and backtesting results are presented. (c) 2006 Elsevier B.V. All rights reserved